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ABSTRACT 

This thesis deals with random processes which are stationary, 
ergodic, and described by correlation functions or power density spectra. 

An attempt has been made to develop a new approach to the study and control 
of random processes which is simple, stresses physical rather than mathe- 
matical interpretation, and is valid when a number of statistically related 
processes are to be processed simultaneously. Among the origianl and fun- 
damental results of this investigation are: 

(1) A closed-form solution is presented for the optimum multi-di- 
mensional system in the Wiener sense. This system operates on n correlated 
random input signals and produces m desired outputs, each of which has 
minimum mean-square error. The solution is dependent upon the factoriza- 
tion of a matrix ^(s) of the cross-power density spectra of the input signals 
into two matrices, such that ^)(s) = G(~s) . G T (s). The nxn physical system 
G(s) determined from this procedure must be realizable and inverse realizable, 
and is the system which would reproduce the measured statistics when excited 
by n uncorrelated white noise sources. 

(2) A general solution is derived for the above matrix factorization 
problem, valid without regard to order, providing the spectra satisfy a 
realizability requirement. The method employs a series of simple matrix 
transformations which manipulate the original matrix into desired forms. 

The key to this solution is a general procedure for reducing a matrix with 
polynomial elements to impotent form, having a constant determinant. This 
latter step is also an original contribution to the theory of matrices with 
algebraic elements. With this solution to the matrix factorization problem, 
essentially no conceptual difference remains between single and multi-di- 
mensional random processes. 

(3) The optimum single or multi-dimensional prediction operation is 
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shown to result from a continuous measurement of the current state 
variables of the hypothetical model G(s) which can create the random pro- 
cess from white noise excitation. These state variables are then weighted 
according to their decay as initial conditions in the desired prediction time 
and the"decayed "output or outputs are the desired prediction. Thus, ex- 
pected behavior of the random process over all future time is compactly 
summarized in the current values of these state variables. 

(4) It is proved that correlation functions measured between two 
variables in a linear system can be viewed as an initial condition response 
of this system. Also, the well-known Wiener-Hopf equation is shown merely 
to require that every error be uncorrelated with past values of every input 
signal. 



(5) If one or more noisy signals have a power density spectra matrix 
(s), which can be factored into G(-s) G T (s), and if G(s) is separated 

such that G(s) = S(s) + N(s), where S(s) and N(s) have signal and noise poles, 
respectively, then it is shown that the optimum filter is a unity feedback 
system with forward transference S(s) N _, (s). This very general result is 
valid for single or multi -dimensional optimum filtering problems. 

(6) A quantitative substitute for the Nyquist sampling theorem is 
presented which is concerned with a measure of the irrecoverable error in- 
herent in representing a continuous random process by its samples. Also, 
the new results in continuous random process theory derived herein are ex- 
tended to the discrete case. 

(7) The concept of "state" of a random process is advanced as 
fundamental information for control use. Two new design principles are 
discussed for the bang-bang control of a linear system subject to a random 
input. In one, suitable for multi -dimensional full throw control, the deter- 
minate Second Method of Lyapunov is extended to include random processes. 

The basic contributions of this thesis are (1) a complete theory of 
multi -dimensional random processes, (2) a simple physical explanation for 
the optimum linear filter and predictor using white-noise generating models, 
and (3) a new approach to stochastic control problems, especially those 
involving saturation, using the concept of the "state" of a random process. 
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CHAPTER I. 



INTRODUCTION 

The word "random" is an adjective which mankind has come to 
use in apology for unwillingness or inability to measure fundamental 
causes for events observed in Nature. Of these events, the random 
process which goes on continuously and indefinitely has captured the 
interest of mathematicians and engineers. There is something com- 
pelling about attempting to describe that which is ever changing, and 
thus undescribable. 

This thesis is concerned with random processes in their sim- 
plest form -- with statistics that do not change with time, and whose 
properties are adequately described by the well-known correlation 
functions. Many able researchers have cleared this path and it could 
well be asked, like an echo from the Second World War, "Is this trip 
necessary?" 

To begin with, a research investigation is generally based on 
aggravation, either with what is not known or with what is known. In 
this work, the latter case is true. It is the opinion of the author that 
the classic and beautiful core theory of Wiener in this area, by its 
very mathematical eloquence, has tended to suppress a more funda- 
mental understanding of what can be known in a random process and 
what cannot. 

In essence, the original work of this thesis starts with the 
well-known fact that the random processes considered here act as if 
they came from a linear system which is excited by the most random 
of signals, "white" noise. This linear system specifies the particular 
random process, and focussing attention on its determinate structure 
is a more satisfying approach, at least to the engineer, than is ac- 
cepting the manipulation of statistical properties of the ever -changing 
output of this system. 

Some of the unsolved problems and prominent possibilities in 
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random process theory which come to mind for possible attack are: 

(1) Conventionally, derivations in the Wiener theory are made 
for optimum systems in the time domain. A pure transform approach 
appears much more desirable. 

(2) A general closed-form solution for the optimum multi -di- 
mensional system has not yet been given in the literature. 

(3) A means has not yet been found for determining a physical 
system capable of reproducing signals with the given statistics of mul- 
ti-dimensional random processes. 

(4) The fundamental results of Wiener theory are the optimum 
predictor and filter. It may be possible that these have a very simple 
interpretation in terms of the equivalent white -noise driven system. 

(5) The correlation functions of many observed random pro- 
cesses have the appearance of an initial condition response of a linear 
system. If this is true, what linear system and what initial conditions? 

(6) What effect would white nd,ise have if suddenly applied to an 
otherwise quiescent linear system? 

(7) There is no valid measure of the inherent error due to sam- 
pling of a random process to replace the "Go-No Go" nature of the 
Nyquist Sampling Theorem. 

(8) If a linear theory produces all the knowable information 
about an input random process, is there some way of intelligently 
using this to control a physical system which has limitations such as 
saturation? No suitable approach to the on-off or bang-bang control 
problem with random excitation has been made which makes complete 
use of this information. 

(9) If a random process is to be examined by means of inves- 
tigation of an effective physical system, can some determinate ap- 
proaches to systems analysis such as the "Second Method of Lyapunov" 
be extended to include random processes? 

This thesis provides a quantitative answer to each of these ques- 
tions or possibilities. The author believes that the results found in this 
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thesis investigation, because of their simplicity and generality, provide 
the most effective means for understanding the nature of stationary ran- 
dom processes. 



CHAPTER H. 



DERIVATION OF OPTIMUM 
SINGLE AND MULTIDIMENSIONAL SYSTEMS 

2. 1 Introduction 

This chapter is concerned with linear systems which operate on 
stationary random processes so as to minimize a quadratic measure of 
error between the desired and actual outputs. In the case of a single ran- 
dom signal, perhaps corrupted by noise, the results of this theory have 
been known for over a decade. Why, then, is it necessary to retrace 
such well-worn steps? 

There are two reasons for this apparent duplication. First of 
all, the author feels that the time -domain derivations found in many 
standard texts of the optimum Wiener filter are unnecessarily compli- 
cated and tend to obscure the basic simplicity of the ideas expressed. 
Secondly and more important, when the optimum system to process two 
or more signals simultaneously is derived, the conventional methods 
rapidly become enmeshed in their own symbology, whereas the steps of 
the single-signal frequency domain approach to be described in this 
chapter allow direct extension to the multi -dimensional case. 

2. 2 Historical perspective 

In this country, the origin of the statistical theory of optimum 
linear systems was the wartime work of Wiener* - . A parallel develop- 

2 

ment in Russia at approximately the same time was made by Kolmogorov . 
The structure of the basic theory was thus well-formed by 1950 for prob- 
lems involving prediction and filtering of a single stationary random pro- 
cess in the presence of additive noise. Significant extensions and clari- 

3 

fication of Wiener's work were made by Zadeh and Ragazzini , Bode 
and Shannon^, Blum**, Lee**, Pike^, and Newton**. The latter's work 
was of particular significance, since it introduced the concept of optimi- 
zation with constraints in order to satisfy certain practical engineering 
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requirements of a system which the basic theory neglected. In the last 
decade, graduate -level control systems engineering texts have gener- 

9 

ally emphasized the statistical approach. These include books by Truxal , 
Newton^, Smith^, Seifert and Steeg^, and Lanning and Battin 

In the multi -dimensional case, the theory is not as well-developed. 
14 

Westcott derived an optimum configuration for the two-dimensional 
15 

case. Amara used a partial matrix approach and successfully derived 
the optimum unrealizable configuration, but his realizable solution was 
only applicable upon very restricted signal conditions. Hsieh and Leondes* 
presented a method for solving for the optimum system involving unde- 
termined coefficients, but the meaning of their solution was obscured 
by the formidable notation employed and no proof of the adequacy of 
their method was offered. 



2. 3 Summary of linear statistical theory 

Figure 2. 1 shows a typical time record of a random process in- 
volving two variables, x and y. The signals to be considered under this 
theory are stationary; that is, they have statistical properties which do 
not change with time. Also, these statistical properties can be approx- 
imated by measurements made on a single long but finite time -recording 
of the particular continuous signal -- that is, the processes satisfy the 
ergodic hypothesis. 




Figure 2. 1 Typical random processes 

The objective of statistical analysis of a random process is to 
detect cause-effect relationships between events -- or signal levels -- 
separated in time. The basic tools in this analysis are the auto -correla- 
tion and the cross -correlation functions. The auto-correlation function, 
y^ x ( / T), is defined as the average value of the product of the instantane- 
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ous signal and the signal level T~ seconds later. 

^xx (r> “ E { x(t) ’ x < t+r )} (2. 1) 

where the symbol — is a defining equality and the operator means 

"the expected value of". Expressed in integral form for the class of 
signals considered, 

Txx <r) J dt x <« ' x <* +T > (2-2) 

=T 



Figure 2. 2 shows a typical auto -correlation function. Note that 
it is even about theT^ = 0 axis, since replacing t by 

XX / XX 

t -T' in Equation 2. 2 does not affect its value. The maximum value of 

^ (T) is at T= 0 for any stationary signal observed in the real world 
xx g 

(a proof is given by Truxal . ) 



The cross -correlation function, ^(T), is defined as the average 
value of the product of the instantaneous signal level of one variable, x, 
and that of another signal, y,T^ seconds later. 



^ xy Cn = E Jx(t) • y(t + r)j (2.3) 

y xy < r > = t^2¥ ^ dt x < t )’y< t+r > < 2 - 4 > 

-T 




Figure 2. 2 A typical auto -correlation function 



In this case, replacing t by t -Tin the integral form yields the 
definition of ), and the peak value of T”’ ) does not necessar- 

ily occur at the origin. Summarizing, 
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(2.5) 






XX 



<-r> 





( 2 . 6 ) 



The auto -correlation functions and all possible cross -correla- 
tion functions among members of a set of random signals completely 
describe the particular process for the purposes of a linear theory. 

One significant use of the auto -correlation function is that ^ (0) 
is, by definition, the mean square value of x. For example, this makes 
it a useful measure of the accuracy of a system when the signal concerned 
is the error. 

Since the correlation functions (for'f'/'O) have the same appear- 
ance as transient signals observed in linear systems it is logical to de- 
fine the Laplace transforms of these functions and inquire as to their 
potential use. As the functions are defined for both positive and nega- 
tive T , the bilateral or "two-sided" Laplace transform is selected for 
use. The bilateral Laplace transform evaluates the positive -time part 
of a signal just as the one-sided Laplace transform does, but the nega- 
tive-time portion has the sign of t changed (ie: "flipped over" the t = 

0 axis), evaluated as a positive -time signal, and the sign of s, the trans- 
form variable, is changed to -s. 

In order to ensure a one-to-one correspondence between the trans- 
form and the time -domain expression, it is necessary to specify that all 
poles in the right half plane (or "negative" poles) correspond to functions 
in negative time and not unstable functions in positive time. 

In this work, the bilateral Laplace transform of the auto and 
cross -correlation function is defined as the auto or cross power density 
spectrum, (^^(s) or <^ X y(s), respectively. The notion of power density 
arises in the following fashion: 



The mean square value of a random signal x is envisioned as a 
generalized form of average energy because of its quadratic nature, and 
is equal by definition to z (0) . If tp (0) is finite, it is equal to the 

* XX / XX 
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sum of the residues of either the left- half or right -half plane poles of 
the transform ]$ xx ( s )> as seen directly from a partial fraction expan- 
sion of ^^(s) an d term-by-term inversion. But by the residue theorem 
of complex variable theory, the evaluation of a closed contour up the 
imaginary axis of the s-plane and enclosing the left -half -plane at infin- 
ity will yield 2vj x summation of residues, providing the contour is of 

the order of no less than y ass — ±oO. That is, ^P^(s) must contain 

s 

at least two more poles than zeros for a finite mean square value, ^ xx ^), 
to exist. 



Thus, 



V (0) = 

I xx 




o O 



(s) 

^XX 



( 2 . 7 ) 



Let s - jco 

f ( 0 ) 



XX 



27 r 




oO 



£ 



xx 



(u>) 



( 2 . 8 ) 



The mean square value (or power) of a signal is thus seen to be 
proportional to the integral of ^ (o>) over all cu, and ^ (u>) quite natu- 

XX XX 

rally is visualized as a power density per unit u>. Most authors have in- 
cluded the in the definition of the power density spectrum so that 
the integral over all co yields the total average power, but this appears 
to be less natural than retaining the pure transform relationship, espe- 
cially since the name "power" is a misnomer in itself. The co notation 
is the most common encountered in past literature on random processes, 
and brings to mind a weighting of harmonic content, considering the ran- 
dom process to be a superposition of an infinite number of infinitely small 
simusoidal waves. 



It might be argued that the choice of nomenclature is a trivial 
matter, but in as much as it influences basic conceptualization of a ran- 
dom process, it is very important and deserves elaboration. 

Ten years ago in automatic control literature, the transfer function 
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of a linear system was invariably written as G(ju), and much was made 
of plots of frequency response on polar or logarithmic coordinates. Fre- 
quency response was almost regarded as an end in itself, and design 

specification in terms of these characteristics helped propagate this 

17 

belief. However, the acceptance of Evan's root-locus method and the 

g 

strong emphasis by Truxal and others towards use of the Laplace 
transform helped unify the differential equation, frequency response, 
and transient response approaches to dynamic behavior of linear systems. 
In proper perspective, frequency response is an often desirable experi- 
mental description of a system and provides, on logarithmic coordinates, 
a rapid means for design of simple control systems when specifications 
on transient behavior are loose. Frequency response is perhaps best 
visualized as an imaginary axis scan on the complex plane, as shown in 
Figure 2. 3, where the function is evaluated by the complex product of 
vectors from all system zeroes divided by vectors from all system poles 
to the particular s = jco point under consideration. 




Figure 2. 3 Frequency response viewed as imaginary axis scan 

Since linear systems were previously regarded in terms of how 
they altered the magnitude and phase of an input sinusoidal signal, essen- 
tially a communications engineering viewpoint, it is natural that random 
processes should have been described in terms of relative frequency 
content. But now that the Laplace transform -- high-lighting the system 
poles and zeroes -- has emerged as perhaps the best index to the prop- 
erties of a linear system, it is necessary to take the viewpoint in a ran- 
dom process that the characteristics of interest are the poles and zeroes 
of the power density spectrum ^ xx (s), and not necessarily the spectrum 
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shape. A useful conception of the spectral representation, as a function 
of u>, is shown in Figure 2.4, where again the magnitude of the spectrum 
is determined as the resultant of vectors from all poles and zeroes of 
$ <s> to the s = ju point. Note that an auto power density spectrum 




Figure 2. 4 Power density spectrum viewed as imaginary axis scan 

has a symmetrical distribution of poles and zeroes, since the relation- 
ship (T~) = P ( = 7") becomes (s) = & ( -s) in the frequency 

xx 'xx XX XX ~i J 

domain which means that, term for term, the LHP poles and zeroes 
must equal the RHP poles and zeroes. 

The basic tools for the examination of random process have been 
presented -- the correlation functions and their transform mates, the 
power density spectra. It now remains to specify how these character- 
istics are altered by passage through a linear system. 

2.4 A general formula for power density spectra transformations 

A derivation is made in this section of a compact expression 
of the cross (or auto) power density spectra between any two signals 
in a linear system as a function of the cross power density spectra of 
the system inputs. This resulting formula will be used consistently in 
this and remaining chapters because of its generality and simplicity. 

The general problem to be considered is pictured in Figure 2.5. 

x and y are two variables (x may equal y) which are the linear responses 

to two sets of random inputs, x. and y., each individual input being oper- 

^ 3 

ated on by a system weighting function, g^(t) or h.(t). The desired quan- 
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Figure 2. 5 General model for linear system 



tity is (I) (s); the known quantities are the cross-power density spectra 

-£ xy 

between any two of the inputs x^ and y.. 

/W AW 

x(t) = ^ g.(t)^f x.(t) l y(t) = h (t)-* y (t) 

1 1 J-/ J J 

where " is a symbolic operator expressing convolution. Transform- 
ing, 



'Vu 



Xls) = ^ G.(s) X.(s) ' Y(s) = 2. HXs) * Y.(s) 

^xy (r) ~ E t ^ X(t) * y(t +T ^ 




assuming that the integration involved with averaging in time can 
commute with the integration of the Laplace transform. The subscripts 
on the operator indicate the time variable which is used in the opera- 
tion. 
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Consider a length of signal which exists for duration 2T, where 



T is arbitrarily large but finite, and which is zero elsewhere. 

^xy <s> * / dt x <t> /y<‘ +T> e S ^”dT 

= T-**o -k _l dtX<t) eSty(s) 

= X( - s) ' Y<s) 

34 

which is a standard result found, for example, in Rice and Solodov- 

.. 35 

nikov 



But, substituting the values of X(-s) and Y(s), 

/W />vV 



Um 1 I 1 G.(-s) 



( s) — 

xy 2T J— ~ x 

J -*•=/ J-i 



H.(s) Y.(s) 
J 3 



■i i 



G.(-s) H (s) ^ X l<- S) Y i (S> 
*=i J-i J 2 T 



/V\. /wc 



G.(-s) H.(s) 
3 



Js) 



i' 1' ' xi yj 
which is the desired result. Several examples will illustrate the con 



(2.9) 



venience of this formula. 



Consider first the system of Figure 2. 6. 



X(S) 



W (?) 



v yc5) 



Figure 2.6 A simple linear system 



X(s) = X(s) ; Y(s) = X(s) ' W(s) 

(s) = W(s) (s) 

xy xx 



( 2 . 10 ) 
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a basic result which has immediate practical consequences. If x and y 
are the available input and output signals of an otherwise inaccessible 
system, the system transfer function can be determined without intro- 
ducing test disturbances by analyzing the cross -correlation between 
x and y. 

Also, 

^ (s) = ^ xx <s) W(s) -vtf-s) (2.11) 

yy ^ 

Next, Figure 2. 7 shows a typical summing operation. 

XiS) > 



yc^ 

Figure 2.7 A typical summing operation 

Z(s) = X(s) * G(s) + Y(s) H(s) 

(s) - $ (s) CK-s) G(s) + <5 (s) G(-s) H(s) + 

^-zz xx -*-xy 

$ (s) H(-s) G(s) + ^ (s) H(~s) H(s) (2.12) 

yx yy 

which is obtained by inspection by performing the necessary cross - 
multiplication and observing the proper sign of s. 

2. 5 Single -dimensional optimum systems 

The classical Wiener theory of an optimum linear system to 
operate on a random process will now be derived using transform ex- 
pressions wherever possible. This clear and direct approach is useful 
in its own right but is basically intended to provide an introduction to 
a similar development for multi "dimensional systems to follow. 

Figure 2. 8 shows the basic configuration to be studied. The 
stationary input random signal v in general will contain a signal to be 
operated on and an extraneous noise component. The ideal signal i is 
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IDEAL 



~U~ 

INPUT 



W(s) 



■ ■+ 



e 

— > 

ERROR 



Figure 2. 8 Configuration of an optimum system 

the mathematical result of some desired operation on the signal compo- 
nent of the input, such as filtering, prediction, or some linear function 
of the signal. Figure 2.9 shows an elaboration of this structure, where 








GjP) 




















JL 


5 




+ /Ov 


V 


W(s) 


v - A 









ns 






Figure 2. 9 Formation of the ideal signal 

the signal component s is operated on by some not -necessarily physi- 
cally realizable transfer function, G^(s), such as 1, e S \ or s. If $ ss 
and are known, and since 

,(s) = <|> G.(s) + § G (s) (2.13) 

d ss d ns d 

^ v ,(s) is as equally valid a statistical description of the desired opera- 
tion as is specification of G^(s). 

The error signal, e, is the difference between the actual response 

of the system to be determined, W(s), and the ideal signal. The optimum 

system will minimize the mean value of error squared, e = ^ ee ^)> 

which is a satisfactory error criteria for many purposes. The use of 

the variance of the first probability distribution of error is a natural 

choice when longtime properties of signals are being examined, as a 

more complex error criterion besides being mathematically intractable 

13 

would require more statistical knowledge of the processes involved. 
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E(s) = I(s) - V(s) W(s) 

§ e( S s ) ~ ~ ^j v (s) W(s) - ^ yi (s) W(-s) + 

$ vv (s) W(s) W(-s) 



The determination of the optimum W(s) in order to minimize the 
integral expression is the standard problem of the calculus of variations. 

If a perturbation in W(s) is made, called a variation, a resulting pertur- 

T 

bation or variation in e results.- More formally, W(s) is replaced by 

W(s) + 6 c fW(s), where 6 is a "small" constant and the variation cfW(s) 

is any allowable change in W(s), or alternately any system which could 

be paralleled with W(s). This restricts Jw(s) to have the properties of 

physically-realizable and stable systems, that is, with no poles in the 

right -half plane. Also, for a finite e , cfw(s) must not be of such order 

2 

as to provide a component of white noise at e when excited by v. e is 
then expanded as a power series in € around £ = 0. The optimum system 
will have been found when the coefficient of the first power of £ is zero 
regardless of the form of <fw(s) -- in other words, no small allowable 
change in W(s) tends to decrease the value of the integral. 

A 



<T7 

J7 



de 

l 

2irj 






■j 

ds 



o 



£ 






“3 



assuming that differentiation may be performed under the integral sign. 

The variational notation will now be shown to follow the usual 
rules for differentiation, considering the individual terms of ^ ee ^ s ^ 
consecutively. 

/ p u <s>] . 0 
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S [$ iv (s) W(s)] = $ v (s) £ W(s) + 6 <fw(sj f= 0 = 5 iv (s, <f W(s) 

cT [? vi (s) W(-s)] = J v .(s) [w(-s) +€ <fw(-s)| = 5 vi (s) cfw(-s) 

^ [^^W(s) W(-s)J = S’ (*) -|^^[w(s) + £<fw(sj] jw(-s) +€/w(-s|j- 

= $&) |Vw(s)W(-s) + W(s)cTw(-s)] 

The only restriction on this analogy with differentiation is that the 
variation of W(s) or W(-s) must carry the proper sign of s. 

JOO 



X 



= 0 = rA- f ds vi. (s)/w(s) - 5 .(s)<fw(-s) 

2irj I iv ^vi 

+ (W(-s) <Tw(s) + W(s)/w(-s)J > 

To simplify this expression, several auxiliary results are needed. 

(1) (f), (-s) = .(s), from the fact that (f. (^F) = .(71, Equation 2. 6. 

* iv **■ vi iv vi 

(2) The sign of s may be changed in any single term of the above integral, 
without affecting its value, since the limit exchange and the sign change 
of the differential ds have cancelling effects. 

Changing the sign of terms as necessary to be able to factor cfw(-s) 
and identifying I). (-s) as .(s) and W (s) as $ (-s) . 



xv 



vx 



w 



vv 



f - 2 

c e 



r 



2nj j 



joO 

ds ■ r .W(-s) f- .(s) + W(s) ) 
L vx x w J 



- 



If the integral exists and the contour is selected so as to enclose 
the left -half plane, the LHP residues must sum to zero for arbitrary 
«fw< -s), which has all its poles in the right -half plane. Obviously, the 
function [$£)W(s)» J <s>] must have no simple poles in the LHP, 
say at s = - a^, for the sum of residues is £ <fw(a.) , an arbitrary 

number for arbitrary <£w(-s). If this function has a multiple pole, say 

1 c th 

., then e>W(-s) could be selected so as to include a m-1 



(s + a)*** 
order zero at s 



a (only poles must be in the RHP), leaving the first 
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order case. Thus, it has been shown that 
have no poles in the left half plane, or 



/ &) W(s) - § .is)] 

L w VI J 



can 



where 



ef .?' 1 [$# W < S >] -&' 1 M 

fr 1 



(2.14) 



11 



is a picturesque operator used by Smith to indicate the 
operation of inverting a transform into its positive and negative time 
parts (iT 1 or the inverse Fourier transform) and using only the posi- 
tive time part in taking the unilateral Laplace transform,oC • Despite a 
possible question as to the uni- or bi-lateral nature of£ , this compact 
notation will be used subsequently to denote the casting out of RHP poles. 

A functional equality of LHP poles, such as in equation 2. 14 
above, is not affected by multiplication of both sides by the same ar- 
bitrary transform having poles in the RHP. For example, J"W(-s) is 

such a function. Now, ^ (s), because of its even nature, can always 

+ ^ w + _* 

be factored into W (-s) <5 (s), where !P (-s) contains only RHP 

w -'-w w 

poles and zeroes and (]5 (s) contains only LHP poles and zeroes. 



■ w 



**' 1 § vv<*> _1 (gy-s) 

F( S )w( S ) 



(fF (-s) 
w 



w(s> = \sT * ^ 

-r w 



5vi< s > 



(2.15) 



This is the desired solution for an optimum system under a 
mean-square error criterion. 

To review the derivation procedure, 

(1) ^ (s) was found using Equation 2. 9. (2)/^ (s) was expressed 

in the compact variational^gptation. (3) d e was placed in the follow- 
ing form: £ e^ = -J' ds 2 cfw( -s) W(s) - $ v .<s)l (4) The 

left-half plane poles of W(s) were shown equal to those of J^(s), 
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and both sides of this equality were multiplied by the inverse factor of 




An example of this procedure is given next to illustrate the ease 

in derivation of extensions to the basic theory. This modification is due 
8 10 

to Newton * and is an attempt to control saturation in a given power 

transducer. Figure 2. 10 shows that a signal driving a fixed element 

G (s) is 
P 




ELEMENTS 



Figure 2. 10 Control of saturation in fixed elements 

2 

to have some linear function (G (s) usually equals 1, s, or s ) of itself 

s 

reproduced as the hypothetical signal c, which will have its mean-square 
value constrained by a Lagrange multiplier as the error is minimized 
in order to control the probability of saturation. 

<f{? + A?} . o - A. L + A <f$ c (|)} 

E(s) = I(s) - V(s) * W(s) ‘ G f (s) 

C(s) = V(s) W(s) G (s) 

s 

= 3T ft) - & 5 )w(-s) G (-s) + &($) W(s) W(-s) G (s) G(-s) 

G G r V 1 X W X X 

W(s) W(~s) G g (s) G g (-s) W ^ S) 

cf <P SV = . (s) G f (s) cT W(s) - $ (s) G„( -s) cf W( -s) 

GG IV I VX I 

+ G f (s) G f (-s) [w(s) cf W(-s) + W(-s) =T W(s)J 

(f $ &) = G g (s) G g (-s) jjW(s)cf W(-s) + W(-s) <Xw(s)"] 

^\e 2 + \c 2 ] = 0 = — f dscTw(-s) [- 2 $ vi (s) G f (-s) 

~JoO 

+ 2 ^&)fG.(s) G.(-s) +XG (s) G (-s)l W(s) 

W [_ I f S S -1 
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Hr' 1 {^)[g [ (s)o 1 (-»)Ao s (»IG b (- s )] W<s)} =// _1 {o f (-s) f vi (s)} 

w(s) = 1 ^ s ) ^^( 5 ) \ 

^ [%0) + A G-^S) 6j(rS) ] + |J>£s) [(^%s) A^CSJ %*j] J 

where the + and - symbols indicate LHP and RHP factors. 

This same result, obtained through standard time-domain tech- 
niques, requires a formidable use of tedious multiple integrals plus the 
complex reasoning behind the time -domain motivation of spectral fac- 
toring. 

It is interesting to note that factoring the input power density 

spectrum, = 1) (s)* ^ (-s), determines the system which could 

w w w 

produce the observed statistics when excited by "white noise" with a 

4 

unity power density spectra, as was pointed out by Bode and Shannon . 

In Figure 2. 11, a white noise signal, with = 1, passes 

through a linear system with a transfer function of (s). ^ &) = 

-j- +• +. w vv 

x w ( s ) iP from equation 2. 11. 



U) 




V* 










Figure 2. 11 Reproduction of observed statistics from white 

noise . 

White noise is a useful abstraction, since it is a totally random 
signal having uniform energy content at all frequencies, or alternately, 
an impulse auto -correlation function. It will be one of the major purposes 
of this thesis to stress the visualization of a random process, single 
or multi -dimensional, in terms of the linear mathematical model which 
could create the process. This has the effect of partitioning the process 
into two parts: (1) The white noise excitation, which is totally random 
and thus unknowable, and (2) The hypothetical physical system, which is 
completely known and which has instantaneous internal signal levels which 
completely define the entire past history of the white noise excitation for 
future use. 
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2. 6 Multi -dimensional optimum systems 



The class of system considered in this section is pictured in 
Figure 2. 12. 




Figure 2. 12 A multi -dimensional system 

Here a set of n input signals, v, each of which may contain a 
signal and noise component which can be correlated with any other signal 
or nojlse is to be processed by a linear multi -dimensional system W(s). 
The n outputs, r, are to be compared with ideal or desired signals, i, 
and the set of differences constitute the error signals. As will be shown 
at the end of this chapter, the ideal signals result from a linear opera- 
tion on the signal components of the input signals, and specification of 

^ J. .ir (S) for j and k = 1, 2, . . . n is enough to uniquely specify 
J ft 

this relationship, as was shown to be true in the one-dimensional case. 

The criterion for optimum performance is that the mean-square 
value of every error signal is to be minimized simultaneously. 

W(s) is best described in matrix notation: 



Ks)] = W(s) irtsf) 



(2.17) 



Where W. .(s) is the transmission linking the i output and the j input. 

th 

Consider the i error signal. 

E.(s) = I(s) - 2. W..(s)V.(s) 



1 X. 

"2 



J-l 1J ^ J 



e. =fe. e. (0) = ^ j ds j£ 

— I oO 1 1 



e. e.(s) 



<fe/ 



rx 



= o = 



2irj 



ds S$e. e,(s) 



l l 



-i°o 
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From equation 2.9, 

'VL 



/n. '•*- 

2) e. e.(s) = Cj) i. i.(s) - Z- ^ i. v.(s) W .(s) - £ ©v. i.(s) W. .( "S ) 
11 - i 1 i 3 ij j=i Ji 13 



J=l 



J 



K=| 



W lk ( S ) 5? v v k (s) 

.th 



f 



In matrix notation, let |W^(s)|be the i row of W(s). The scalar 
e. e^(s) is then seen to be expressed by 

e. e.(s) = ^i. i.(s) - W.(s) ^Pi. v.(- s ) - W.(-s) ^"v. i.(s) 

ii ii i I i 3 J I i i 3 i J 



+ W.(-s) 
I i 



$ w (s) ] W i (s) ] 



Let|W^(s)j be replaced by |W^(s)^+ €|tfw^(s)|, whereC is a scalar 
and the variation^ W^(s)| is an arbitrary row vector, each element of 
which satisfies the same physical realizability condition as in the one- 
dimensional case. 

<f $ 

6 i e i^ s ) will be evaluated term by term to show that the 
matrix variation is found by an analog to matrix differentials. 

<T $i. i.(s) = 0 



«f | |W i (s) | i. vXs) j* 



1 1 



"a ivW,(s) ,+ 6 cfw,(s) \ ' i. v (s)| 



oe \ Li J | 1 



e = o 




f v. i. 



5 V>J 




= I Jw.(-s) | J v. i.(s)J 



€=o 



|( | w i<- s >, / w j<- s >^ • [$ <&] • { W^s] + 6<rw.(s>ip - 

^ € = o 
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= iW.(-s) , 

1 Jl W 



e. 

1 



2irj 






w]/w.(s)] + | /w.(-s) [gCo]w.(s)] 

06 r 

ds j- i/w.U), fij v.(s)J - /w.t - 5 ) ^ V. i.(s)j 

, w ^, r^H + / w i<- s > r^i w i< s > 



Each term under the integral sign is a scalar and can be trans- 
posed at will, and the sign of s changed as was described in the single - 

dimensional case. Also, (fr"*"(-s) = ^ (s) since v.(-s) =JJv. v.(s), 

-^w w 3 1 ^13 

Equation 2.6. 



ST? 



= 0 = 



k J ds i <fw i ( ' s) i \' ^ ‘i v j ( " s !l ‘ $ v j ‘i (s) ] 

+ [$w<- s >) w.(sj] +[$&]w.<s)]l 



2 k J * 2 



fv. i.( S )] + [^]w.< S ) 



-JOO 



This scalar integral expression is identical with the sum of n 

one -dimensional cases and the same reasoning, element by element, can 

be applied to the column vector as was applied to the single dimensional 

case. That is, there can be no net LHP poles in any element of |^^j|w^(s)| 

-(£v i (s)j since they are separately multiplied by arbitrary functions 
J ^ 

having RHP poles only. 



Therefore, 



{[M W i(S »]) - Sf' 1 ^ Ms)]’ 



1,2 ...A\) 



where the^i^ * operator is understood to apply to each element in the 
cblumn vector. 

An expression involving the matrix Jw(s)jis thus found: 



& £ ' 



-1 



[$w (s >] [wTsi]} - y y 1 v. i i(8 )]} 



The remainder of this work will need to express compactly the 
cross -power density spectra which exist between signals in two sets or 
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vectors in a random process. The following convention will be observed. 

^ (s) will have an i.^ element (s). Thus 

xy j x i y i 

t ir -1 wT<s> ) vi (s) } <2 ' 18) 

This is the implicit solution to the optimum multi -dimensional 
system under mean-square error criteria. In the special case of a 
single -dimensional system, the result is identical to that derived previously 
in Equation 2. 15. 



Unfortunately, W(s) is not directly obtainable from this expression 

since contains poles in both the LHP and the RHP. This defining 

equation implicitly involving W(s) has been previously obtained inde- 

1 5 — ^ g 

pendently by Amara and Hsieh and Leondes , and the next section 
will outline and analyze their proposed methods of solution for this set 
of intercoupled equations. 



2. 7 Past attempts to determine optimum multi -dimensional system 



Hsieh and Leondes employed time-domain concepts in deriva- 
tion of the optimum system. Their solution will be expressed in the matrix 
notation of the previous section. The basic problem is to determine W(s) 
from the equation 







(2.18) 



Hsieh and Leondes added an undetermined matrix F(s) to the 
above equation so as to provide an equality of both LHP and RHP poles. 




(2.19) 



F(s) contains the RHP poles of ^^(s) W (s) C^) 

Thus the a 1 operator is no longer applicable, it being understood that 
W (s) will have no poles in the RHP. 



W T (s) = §> F(s)1 

W ( VI j 
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T 

W (s) = 



Adj 35 (*) 

VV 






F(s) 



where A + and A are the LHP and RHP factors of I ^ (s) 



and j^adj <|HS,fjis the adjoint matrix of ^ vv (s) . 



vv 



( 2 . 20 ) 

respectively 



+ T 
A W (s) = 



T — [ A 3 ^ v 4 SJ ]^ vi (s) + ^>1 



F(s) 



Let -L [Adj S v ^]? v .(s) 

A * — . — 



= H + (s) + H (s) 



( 2 . 21 ) 



where H (s) is known and contains only LHP poles, obtained by perform - 

A 



ing a partial fraction expansion of each element of — J^Adj 
H (s) contains only RHP poles. 



Each element of 






can contain only as its LHP poles the 



LHP poles of . — 2 ^=. F(s) will contain only RHP poles. Thus, 



vv 



[ /**'• 

Adj I? (S) F(s) = ^ -i- _ c 1 + J 

J w J s + P. 

A~\ 1 



(s) 



(2.22) 



where -P is the i LHP pole location of having an undetermined 

matrix coefficient C 1 , and J (s) is a matrix with only RHP poles which is 
not considered further. Accordingly, 

£ 

s 



W (s) 



1 

A* 



H (s) 






(2.23) 



At this point, it is claimed by Hsieh and Leondes that the un- 

x 

determined matrix coefficients can be obtained by substituting W (s) 
into the basic equation, 2. 18. 



ii ’ {fjJ • -a*- 



/)*V 



H*(s) + £ 



I 




-I£ -W v .(s) 



(2.24) 



No proof is offered as to the sufficiency of the resulting equations. 

The non- generality of this method will now be demonstrated by 
considering a particular example, a multi -dimensional predictor, and 
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showing that the resulting equations are insufficient to determine the 
C 1 coefficient matrices. 

A multi -dimensional predictor example 



The input signals, v., have no noise superimposed, and are re- 

presented by the known matrix (p C s >. The i ideal signal is a prediction 
th 

of the i input signal T - seconds in the future. 



V.(s) = V.(s) ; 



I.(s) = e s 1 "v.(s) 



W . i.(s) = e ST ^ . v.(s) 

vl j vi j 

^ .(s) = e ST ^ (s) 



VI 



W 



From equation 2. 21, 



h + <s> -JJr' 1 


P 


~ Ad J 5" 


=Ir 1 


( A+ 


5r'(s) . 

x V v 


-w 1 


( A+ 




From equation 2. 23, 


w < s > = I* 


{ V A 


-1 + ST 

(A e 



■vv 









^•1 



where C is determined from the equality of Equation 2. 24. 

U~{ u > • ± i‘ r I)+? ( 4 ijpp 

Next it will be shown that a partial fraction expansion of 
la 1 ' in the poles of ^ 0^ is equal identically to the ex- 
pansion of <s) "j. . This will be done by proving 




that 



r 



(a + (s) ^ iT ) 






-P, t- 



5= -P 



A* (*) 

which ensures that the external factors outside both the (£) matrices 
are identical for each pole. 



It is assumed for simplicity, and since this example is designed 
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to 



be essentially a counter-example, that only simple poles exist in A + (s). 

s M. = Jw 

c " ©»> Jr (st Pi) 



-'•*1 



=. 1 1 fPl*) c ^ 1 = ^PHiK 

JT (St PS) j (s+ pj) ^(P-i-Pi) 

i/-' (4%) 



A + ts) 



IT (s t P*) (f Pfa)C 



S=-P. 



, /Wv 

(i 



-R.T 



P(s) 

= 1 7T (,♦*) 



(s*Pj tt . (?*-&) 



l =, ^ PH) 



s=-p, 



•OC C-i ; 



Kjr - ' J 

Each term of this summation equals zero when X. it J 

I = £tPj) e' Pj ' f ' rr f P*. - Pj J . P - Pj t 

ls =-?j fijh-Pj) PtPj) K " “ 

Therefore, equations involving the LHP poles of which 

can determine C 1 , are 

JX 1 







= 2 _0_ 
-'= / 



s’ + P. 

l 



A + (-P.) 



(-P.) c 1 

W 1 



Jr-p, 



A + (-P.) ^vv =I V 



(i = 1, 2, 



. m) 



where 0 is the null matrix. 

1 



The matrix 



^ (-P.) will have non- zero values only 



A + (P.) J-w i 

in elements where the scalar $~v. v.(s) has a pole at s = -P. . Since 

i J i 

simple poles were assumed in (s) , and since a determinant is 

formed with each separate term containing only one element from each 

th 

column and row, it is clear that the i LHP pole will in general lie along 
one column or row of ^^(s) (actually a column, as will become clear 
in Chapter 3). Thus, an equation of the form 
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O O ' * * • o 


O * V . - 




o 


; • . . . * 


• (T = 


i * * * # 

• ♦ 


• • ■ * 




* • ♦ 


» * • ► * * 






o • • • 0 




o * * # *■ * o 



is patently not enough to determine C 1 . 

Through the medium of an example involving a multi -dimensional 
predictor, it has been demonstrated by counter-example that the proce- 
dure of Hsieh and Leondes is not generally applicable. A method of un- 
determined coefficients is only valid when it can be proven that the co- 
efficients can in fact be determined. 

15 

Amara approached the same problem, but attempted to find a 
closed-form solution and was successful for a quite restricted class of 
multi -dimensional random processes. Unfortunately, in his derivation 
of the optimum system he chose to minimize instead of the mean square 
error of each output the mean square value of the total sum of all the 
errors, which could allow undesirable cancellation effects between the 
individual errors and in general is not the best quadratic error criterion. 
It is interesting to note that his implicit solution is identical with that 
obtained by considering each error separately, as in section 2. 6. 



Amara considered the class of random processes characterized 
by a matrix of power density spectra, (£> ^(s), which can be transformed 
to a diagonal form by pre- and post -multiplication by matrices with nu- 
merical elements, such that 



U 



f (s) 

w 





where £ is the Kronecker delta, ( <4 . 

ij ij 



(s) 

w 



= U 



i i j; cT. . =i, i = j) 

rD...(s)ii fu 7- ] * i 

L IJ IJ i 
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+ + 

D. (s) = Di.(-s) D!.(s) 
ij 13 iJ 

Thus, the optimum system is given by 



JX _1 {-H- 1 w T ( S )| 

1 {C D i/- s) ^] £#■><* *«] K ]' 1 wT(S> } fvl^ 

If [u T ] 1 W (s) is considered as another optimum system, the 



above equation in similar to n one-dimensional optimum systems, and 




The requirement that the power density spectra matrix be diago- 
nalized by a numerical matrix is a severe limitation on random processes 
in general, as wili become more clear in Chapter 3. 



In summary, there is no hitherto published satisfactory solution 
for the optimum n-dimensional system. The next section will consider 
a more general approach to this problem, which will yield physical insight 
into random processes and bypass the restrictions of the previously des- 
cribed methods. 



2. 8 A new closed-form solution for an optimum multi -dimensional system 

In the solution of the single -dimensional optimum system, where 
from Equation 2. 14, 

^ _1 fw< s > • wis) } vi (=> 1 [ 

( 5 * (s) was factored into RHP and LHP terms 

* w 

3? (s) = (j) "Vs) (s) (2.25) 

w w X w 

and both sides of the equation were multiplied by , maintaining 

x vv 



- 28 - 



theJJr 1 equality. 



If the matrix ([) (s) could be factored into two matrices. 



w 



(s) = <5~ (s) . Cf* (s) 

w - 1 - w W 



w 



w 



where ^^(s) and its inverse contains only RHP poles, it is logical 
to inquire whether multiplying both sides of the*^ * matrix equality 



w 



(s) 



-1 



would preserve this identity. 



More generally, if the matrix equation is given 
1 ^ A(s) 1 1 {* B(s)} 

d ° eS 1 C(s) A(s ) 1 1 C(s) B(s) ~| 

th 

where C(s) has only RHP poles in every element? The ij elements of 
C(s ) A(s) and C(s) B(s) are, respectively, 

'y C... Ak. and S C„ Bk. 

£| ^ J fea", lk J 

From the previous arguments of this chapter, 

IJ- - 1 { C. k Ak. } - j* - 1 ( C. k Bk. ] 

since 

^ } = ** _l { Bk ji 

Obviously, the addition of n equalities of LHP poles is still a valid 
equality. 

Thus it has been demonstrated that multiplying a matrix 1 

equality by a matrix with all poles in the RHP preserves the * equality. 

** -1 fe>>T' K>> K v (s > w T (s) j 

_1 * V<S > WT(S} J = fvv <S > wT(s ) 1 ^>} 

W T (S> = [0*>] _1 3 * |[$' vv (s)] _1 § vl (s) ~l (2. 26) 

In the above steps, ^ (s) * must contain only RHP poles, to 

justify the operation under the^^ * operator, and (s) must contain 
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(2.28) 



only LHP poles, to justify the removal oi33r 

Further restrictions must obviously be placed on ^ vv (s) and 
$ (s). It has been shown in Section 2. 6 that (J5 ^(s) = ^(-s) 

Therefore, let (35 (s) = G(-s) . G T (s) (2.27) 

— j W — -- - - - 

where G(s) and G(s) are both physically realizeable. Thus 

w T <s ) = fq T (a>l -1 ^' 1 {r G <- 3> ]lvi (s >l 

In section 2. 5 it was pointed out that factoring the single dimen- 
sional 3T (s) into (R" + (-s) . 3T (s) determined & (s), the trans- 

fer function of a linear system which could reproduce the observed signals 
when excited by white noise with unit power density. This is the Bode- 

4 

Shannon approach . It is natural to inquire if a similar interpretation 
can be placed on the factoring of ^^(s). 

Suppose a set of n uncorrelated unity white noise excitations, w., 

3 

are applied to a physical matrix filter, G(s), as shown in Figure 2. 13. 




V.(s) = 



Figure 2.13. A random process created by n white noise sources 
G..(s) W.(s) 



J= I 






3 



A*- 






<3>V v (s) = 5 S G,Js> 5 W 1 

J -<=/ U til J 1 



w 



but $ Wj 



w k - 1 

= 0 



/VU 

^ V V.(s) = 

1 J J--, 



1 = k 
1 / k 



G u<-») V s) 



w 



In matrix notation, 
<s) = G(-s) G T (s) 



(2.27) 
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which is the desired result. That is, the process of matrix factoring, 
which leads to a closed-form solution to the optimum multi-dimensional 
system, is identi cal to the problem of finding a physical system which 
can produce the observed statistics with white noise excitation. 

Thus, the multi -dimensional problem has been shown to parallel 
exactly the single -dimensional case in notation and meaning, if the ma- 
trix expression is substituted for the one -dimensional transfer function. 

Chapter III will present various approaches and a complete solu- 
tion to the formidable matrix factorization problem. It should be pointed 
out again that this matrix approach produces the first general closed-form 
solution to the optimum multi -dimensional system in the Wiener sense. 



2.9 Statistical transformations on random vectors 



A great similarity has been demonstrated between the scalar 

and the matrix representation of random processes. For example, & xx (s) 

describes a single random process just as ^ (s) describes a set or 

"vector" of n random processes. Some of the simpler relations to be 

18 

derived were earlier presented by Summers , but in view of the sim- 
plicity of derivation using equation 2.9, they will be repeated here. 



Consider first the simple configuration of Figure 2. 14 

y 



Figure 2. 14 



Multi -dimensional System 




Y(s)| = G(s) X(s)j 

/yv 



Y.(s) = 



G <s) X.(s) 
^ J 



J= I 

y -i y i (s) = jr, G il ( ' s> ^ V s> ^ X 1 x k <s) 

<s> = CK-s> § (s) G T (s> 

yy xx 



(2.29) 
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In the special case where xj is a set of uncorrelated white noise 
signals with unit power density. 




£ yy g> 



_I_ 

G(-s) G T (s) 



verifying Equation 2.27. 



/n. 




Next, the summing operation of Figure 2. 15 is examined. 






GCS) 



H (A) 






2 



Figure 2. 15 A Multi -Dimensional Summing Operation 

I <?> /k< s J 



J-/ 



ft* I 



-c- -"A. /*A 

J-l fe=l 

+ I, ^ £, ^ <*> 



/K 



+ £^ teC ' s) J, % a / s ; 

^ (s) = G(-s)$'U) G(s) + G(-s) 3F (s) H T (s) 

7 7 — ** YY YV" 



+ H(-s) $ (s) G(s) + H( -s) 5 (s) H T (s) 

yx yy 



(2.31) 



The preceeding configurations were examined in deliberate 
similarity to the scalar results of Section 2.4. It inferentially appears 
that a general formula for vector random processes can be expressed 
just as Equation 2.9 applies to scalar processes. 
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(2.32) 



G.(~s) 



x. y (s) H. T (s) 

J J 



AV />f\ 

£*y‘ S > ’ 2 f, 

I J =• J — — ^ 

where the vector X(s)| = i£ G.(s) X.(s) and Y(s)l- ^ H.(s) Y.(s) . 

-jU -*«i -- 1 1 J th J >i - J J _J 

The ij subscript of 3P x. y.(s) refers to the i vector input, x. , making 

— ^ — th 1 

up x , and similarly for the j vector excitation of y . 

To prove this formula, which is believed to be the most general 
expression of statistical transformations in linear systems, consider the 
system of Figure 2. 16. 



5 




MATRIX 

G. (s) 

H. (s) 
j 

X.(s) 

Y(s) 

J 

X(s) 

Y(s> 



ELEMENT 

G i pq(s) 

Hi tu (s) 

x >> 

Y i u (s) 

X (s) 

P 

Y t (s) 



Figure 2. 16. A general multi -dimensional system 
/n. , r 

X (s) = 

P 



Y t (s) = 



l ■ <") 

I (1, Y ‘.“) 



From the basic equation, 2.9. 

^ • r /vw 



X P y ‘ <S> = j?, ^ ( G W' S) ) 
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/W- 

2 

-*.= | J-l 

± £ 

| -J-l 

/W /W^ 



® ,., . 2 2 



•^=1 J - 



r r 

2 Gi p, ( ' S) ^ H \u <s) ® X ^u <s) 

_ *!- I M/li / T 

p 4 t element o {gVs) |xjj(s) [H j (s)l J 

G 1 (-s) § x 1 y j (s) [H^s)] 1 



(2.32) 



With the use of this formula, statistical relationships in multi- 
dimensional system variables are swiftly expressed. An example will 
prove the previous statement that ^ .(s) is a sufficient description of 
the ideal signal in multi -dimensional optimization. Consider Figure 2. 17, 




where all variables are random vectors and all systems are matrix 
operators. 



v(s) 


= S(s)J + 


N(s)_ 


I(s)" 


■ [ G d (s) J 


S(sf 



$ vi U> * $ ss <s> G>) + $ ns U> G^fs) (2.33) 



Thus, $vi< s > is equivalent to G^(s) if the input statistics are 

known. 
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CHAPTER HI. 



MATRIX FACTORIZATION 



3. 1 Statement of the problem 

This chapter is concerned with factoring a matrix of cross -power 
spectra between signals in a multi -dimensional random process. Chapter 
H has shown that solution of this problem will yield two significant results: 

(1) A closed-form solution can be found for an optimum multi- 
dimensional configuration in the Wiener sense. 

(2) A multi -dimensional linear model is determined which can 
reproduce the observed statistics when excited by a number of uncorre- 
lated white noise sources. 



The basic equation is 



(s) = G(-s) G T (s) 



w 



(2.27) 



or, in expanded form. 



G n<- S > 

g 2 i ( -s) 



G nl ( - S) 



v 1 v 1 (s) 



3? V 2 V 1< S ) 



^"v v.(s) 
n 1 



G 12 ( ‘ S) 



• • 



V 1 V 2 <S) 



G ln (-s) 



0 • • • 



• • • • 



nn 



V V (s) 
1 n 



© • 



V V (s) 

n n 



G u U> g 21 (s> 

Gi 2 (s) 



G ln< S > 



. . G nl (s) 



• • * 



• • 



© • 



. G (s) 
nn 
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The G(s) which is found as a result of the factorization process 
is the matrix filter described in (2) above. Each element of G(s) and 
G(s) l must be physically realizable in order to meet the requirements 
given in section 2. 8 for the solution of an optimum multi -dimensional 
configuration. 



3.2 Realizability considerations 



Before plunging into a solution of this thorny problem, it is nec- 



essary and useful to examine the properties of Jis) which characterize 
a set of random processes which could actually be found in the real world. 

The ii^ 1 element of (s) is ^f) v.v.(s), where v. and v, are mem- 

w 1 1 J 1 j__ 

bers of an n-dimensional random process. Since $ v. v.(-s) = v. v.(s), 

T T" T ^ J J f 

(-s) = (s). 

* W vv 



.19 



In addition, Kraus and Potzl have proven that a necessary 
and sufficient condition for ^(s) to represent a valid multi -dimensional 
random process is that ^ (ju) be positive definite for all w. This arises 



w 



quite naturally if the n signals are allowed to pass through a system G 
which multiplies each signal by an arbitrary constant and sums the total. 
The power density spectrum in w of the single output is, from Eq. 2. 29, 



G__ g] 



This spectrum must have a non-negative value for all values 

of w, since a negative mean square value of power density cannot exist. 

Thus ^ (iw) must be non-negative definite for all values of w. The 
vv . 

special case where |^ vv (j«) J equals zero for all values of w will be con- 
sidered separatedly in section 3. 9, and a positive -definite limitation on 
$ (j«) will henceforth be considered a valid demonstration of the rea- 

lizability of the random process. As will become more clear in the re- 
mainder of this section, the only other case where a zero value of power 
density can occur at a finite value of « is the occurrance of a multiple 
even-order zero on the jw axis in ^(s) | . 
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Positive-definiteness is a property of a matrix which is capable 

of a number of separate verifications. For the purpose of this theory, a 

20 

particular method indicated by Bellman is preferable. He states that 
a necessary and sufficient test for positive -definiteness of a Hermitian 
matrix is that each of the diagonal elements be positive and that the de- 
terminant be also positive. In the power density spectra application, 
this criterion means that the power density spectrum of each of the n 
random signals must be positive, as well as | ^(jw) | , for all w. 



It is interesting to relate these requirements to known properties 
of the auto and cross -correlation functions. For simplicity, the 2X2 case 
will be examined. 



5 





The requirements for realizability are that “), 5 22 (j“), and 

^ 22 U W ) " 3? l2 (j“) ^ 12 ( each be S r eater than zero for allco. 




Newton, Gould^and Kaiser have presented some physical rea- 
lizability requirements on the correlation functions, derived from ini- 
tially setting the square of a linear function of the signals equal to or 
greater than zero: 



^(jl (°) > ^a(T-) (U = 1,2.) (3.1) 

Yu <°) % iL o) > l^ l lr)l z — ~o < "T- (3> 2) 



A relationship between the power density spectra and the corre- 
lation realizability requirements will now be derived. Eq. 3. 1 can be 
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expressed for i = 1, as 



2 7TJ 



ds 



fllO 



2 7T j 



ds e S ^ ® 



— I cO 






Replacing s by ju. 



^ du (1 - e**“^ ) > 0 



Since the real part of 1 - e J is always equal to or greater than 
zero, this integral will always be greater than zero if £ 0 for 

all w. This relates the positiveness of (P^joj) (or ^22^°^ ^ to * a<rt 
that a signal has the highest correlation with itself as opposed to any 
time -shifted version of itself. 



At this point, it is well to ask if the positivity of ^^(s) can 



determined by inspection. It is not enough that I), ,(s) = , ( -s). For 

exam pl e, $ n <s> = ( _ S ^ W + 2) 

-to + 3 



11'“' -*11' 
satisfies this relationship but 



u' 2 + 4 



is negative for co >|3". 



The example above contains a conjugate pair of zeroes on the jco 
axis and is not factorable into (^.^(s) * This pair of simple 

zeroes are the only factors for which ^^(s) = ^^(-s) and which cannot 
be factored with mirror symmetry about the jco axis. Thus, factoriza a 
tion is the only realizability requirement for a single power density spectra. 



The second correlation function inequality, Eq. 3. 2, may be 
written as 

± J Js, $„(!,). J- _ 



4 p, 3U,;e s ' r . > o 

_JoO iT] J J 



or, replacing s by jcj. 



->o 
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_<rO — <>0 

The maximum value of e^ W l W 2^ $ „«“!> * 12< "^ W 2 ^ occur 

when = w and | 12 (jw)| is a maximum * Therefore, the minimum 

value of the integrand is 

3> u <;f) J 22 ^ w ) ~ 

for some value of u. If this integrand is positive for all gj, which is the 
previously given realizability criterion, the integral will always be pos- 
itive. Positivity of this integrand is again equivalent to factorizability 
of the 2X2 I ^ yv (s) | , as was true for the auto-power density spectra. 

In summary, the realizability criteria found in the literature for 
the existance of the correlation and spectral functions of random processes 
are related and the factorability of the individual power density spectra 
and the matrix determinant is enough to satisfy all requirements. The 
reason for the emphasis on this matter of realizability is that any method 
of finding a real system which can create the observed statistics must 
fail when either the diagonal elements or the determinant of ^ vv (s) 
cannot be factored, for otherwise the paradox of unrealizable signals 
being created by a realizable system would exist. 

3. 3 Two special cases 

In this section, two special matrix configurations will be examined 
which can be readily factored. These particular cases are of importance 
since they provide goals for a more general factorization procedure. 

When ^^(s) is a diagonal matrix, each element must be able to 
be factored into LHP and RHP terms, as shown in section 3. 2. There- 
fore, 

§ w (s) = D(-s ) D (s) 
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where D(s) is a diagonal matrix containing the LHP factors of all the 
diagonal elements of ^^(s). 

The second example of an easily factored matrix is the numerical 
21 

Hermitian matrix. Lee has investigated this problem and has proved 

that a solution always exists providing that the matrix is positive definite. 

T 

The problem is to factor H into X . X , where X is a numerical matrix. 



Lee shows that a canonical triangular form exists for this problem, 
T 

H = T . T , where T is triangular and an entire family of solutions is 

T T T 

generated by T . U . U . T where U . U = I, or U is a unitary matrix 
with real elements. In illustration of this result, suppose H = (13 5 

5 2 



and 



T = 



11 



12 



22 



The elements of T can be solved for consecutively because of the 



triangular form, yielding 


col 

o 

1 






T 


5 1 




1 


H = T . T 


fn m 




1 

o 



A general form for a 2X2 unitary matrix is 

f\-~C C 



U = 



a 



-I < <L C 



J 



(3.3) 



This single degree of freedom reflects the difference between 
the number of unknowns, 4, and the number of independent equations 
which can be written, 3 (as the symmetrical form of T leads to identical 
equations for transpose pairs off the main diagonal). In the general case, 
— bounded variables can be adjusted independently in the factoriza- 
tion problem. 



The particular significance of the numerical case is that the 
general factorization procedure to be presented in section 3. 5 will re- 
duce in the last stage to a matrix with only numbers. Another perhaps 
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more conceptual use of this special result is to visualize a matrix a? (ito) 

w 

as a Hermitian matrix which can be factored for every value of to, pro- 
viding that the matrix remains positive definite (the realizability require- 
ment), and thus a matrix which is some function of w does exist. 

It might seem at first approach that a triangular form could be 
postulated for ^^(s) factorization, in analogy to the numerical case. 

This is unfortunately not true, as will be demonstrated below. 



Referring to the general two-dimensional case. 



$ U <S> 


f 12 ( s) 


I 


G u (-s) 


0 




G^(s) 


G 21 (S> 


I 12 <-S> 


£ 2 2< s > J 


1 ' 


_ G 21<- S) 


G 22 ( ' S) 




0 


G 22 (s> 



G u (-s) G u (s> - I u (s) - $>> $>> 



Suppose that 

G^(s) = ^ j^(s) 

G U <S > G 21 ( - s) ' ®12 ( - S) 

G 21 (s> ' ? 12 <S> 



If G^(s) has its zeroes in the LHP, G_j(s) will have these as 

poles in the RHP. If G. -(s) had been selected to have RHP zeroes and 

11 -1 

LHP poles, the inverse matrix G(s) would be physically unrealizable. 



G(s) 



-1 



G n ( s) 

°21 (s> 

G n (s) G 22 (s) 



0 

1 



Accordingly, the triangular form does not yield both a solution 
with a realizable and inverse realizable G(s). However, it offers a use- 
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ful method of reproducing a multi -dimensional random process in an analog 
computer where inverse realizability is of no concern. To assure a rea- 
lizable G(s) , the elements of which may be solved for successively, it is 
only necessary to select the diagonal elements of G(s) with RHP zeroes 
and LHP poles. 



3. 4 Properties of matrix transformations 

The next section will present a general method for solving the 
matrix problem 

$ (s) = G(-s) . G T (s) (2.27) 



w 



The philosophy of approach will be to multiply ^^(s) by a suc- 
cession of simple matrices, transforming it at every step, until the nu- 
merical form is reached. In this section, the properties of simple ma- 
trix transformations will be presented, emphasizing the viewpoint that 
a matrix multiplication can be used as a tool to mold a given matrix 
into a desired form. 



There are three basic matrix manipulations to be considered: 

(1) Multiplying a row by a function of s and adding it to another 

row. 

(2) Multiplying a row by a function of s. 

(3) Exchanging rows. 

In the above list and in the discussion to follow, operations on 
rows by premultiplication are investigated. The results are equally 
applicable to column operations through post -multiplication, however. 

First, any row operation on a matrix can be accomplished by 
premultiplying the matrix by an identity matrix on which the desired 
row operations have been performed. The properties of interest in these 
transformations include the value of the determinant of the transforming 
identity matrix, and the realizability and inverse realizability of this 
matrix. In this particular application, as will be shown in the next sec- 
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tion, row operations are performed with matrices whose elements must 
have only RHP poles and whose inverse must also only have RHP pole 
elements . 



(1) Multiplying a row by a function of s and adding it to another row. 



T( -s) 



1 0 0 

0 10 

0 0 1 

A(-s) B(-s) C(-s) 



0 

0 

0 

1 



The above matrix multiplies the first row by A(~s), the second 
row by B(-s), and the third row by C(-s), and adds the total to the last 
row. | T(-s) I = 1. 



-1 

T(=s) 



1 
0 
0 

-A( 



0 

1 

0 

-B(-s) 



0 

0 

1 

-C(-s) 




The simple form of the inverse will result for all matrices which 
add to or from only one row. If A(-s), B(-s), and C(~s) have RHP poles 
or no poles the matrices are proper for this application, regardless of 
the location of the element zeroes. 



(2) Multiplying a row by a function of s. 

0 0 0 

1 0 0 

0 10 
0 0 D(-s) 

The above matrix multiplies the last row by D(-s). T(s)| = D{-s). 



T(-s) = 
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-1 

T <-s) = 



0 

1 

0 

0 



0 

0 

1 

0 



0 

0 

0 

1 



D(-s) 



D(-s) must have both RHP poles and zeroes to be a proper trans- 
formation. 

(3) Exchanging rows. 



1 

0 

0 

0 



0 

1 

0 

0 



0 

0 

0 

1 



■1 



The above matrix exchanges the third and fourth row. | T | = -1. 



= T. 



The matrices described above perform simple transformations, 
possess simple inverses, and in the second case can modify the deter- 
minant of the transformed matrix by other than a constant. 

3.5 Matrix factorization: A general solution 

A procedure is to be described in this section which will always 
yield a solution to the matrix factorization problem regardless of order, 
providing realizability criteria are satisfied. Because of the complexity 
of the problem, no easy solution appears to exist. However, the method 
of factorization to be presented here has been broken down into several 
separate phases with each phase consisting of simple matrix transforma- 
tions and each having a well-defined goal. 



Each transformation step can be presented in the following fashion: 

i - 1 „ i, 

T. 



M-s) . (s) . T. Is) = 



(s) 



(3.4) 
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The relationship between the pre and post-multiplication matrice 
is specified in order to ensure that (j5^(-s) = *(s)J ^ for all i. 

The overall objective of this procedure is to produce a matrix 
with numerical elements, ^ , after a number of successive transfor- 

mations. Thus, if Q) (s) =: (5°(s) 

w 

T r (-s) . . . . T 2 <-s) T^-s) §"<s) T^ts) T 2 T (s) ....TjJ(s) 

. 

T 

can be factored into two numerical matrices, N . N 4 Inverting, 

$„(■) = §°(s) = T^-s) . T 2 (-s) ...T r ~* g) . NpT^^s) . T 2 "(s) . . . t/s) 



= G(-s ) . G (s) 

G(s ) = T 1 ”( 1 -s) . T 2 ~(s) . . . T r "(s ) . _N 



G(s 



i - 



N 



-1 



T^(s) . . . T 2 (s) . T x (s) 



(3.5) 

(3.6) 



A proper solution of the problem will yield a physically-realizable 
G(s ) and G(s) . This will obviously occur if T^(s) and T^ (s) have LHP poles 
only for all i. In other words, as (^(s) is manipulated into various con- 
figurations the realizability requirements on the solution will be met if 
each transforming matrix meets these requirements. Drawing on the 
results of section 3. 4, the following constraints exist on the elementary 
matrix transformation T.(-s) : 

l 

(1) If T^(-s) multiplies one row of with a function of s and 

adds it to another, this function must have no poles in the LHP. 

(2) If Tj(-s) multiplies a row of ^ tsf with a function of s, this 
function must have no poles, or zeroes in the LHP. 

Since the equation 

i 7 L ~ T, 



5 X (s) = T.(-s) (g X (s) T. (s) 



is in the form of equation 2. 29, T^(s) can be interpreted as a physical 
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suc- 



system with an input random process having a matrix of cross power 

density spectra ^ * \s), and with an output spectra of C^^s) . The 

cession of matrix transformations then is equivalent to cascading a 

series of physical systems until an output spectra involving only white 

T 

noise -- the numerical matrix N . N_ - is achieved. This white noise 
random process is operated on by N to produce a unit-valued uncorre- 
lated set, whose spectra is given by the identity matrix. The total cascaded 
system thus operates on the given random process and produces uncorre- 
lated white noise, and is naturally envisioned as the inverse of the hypo- 
thetical physical system creating the random process. 

There are three general phases to this matrix factorization solu- 
tion: 

(1) Pole removal. The pole removal phase starts with the given 
matrix and removes the poles of every element. 

(2) Determinant reduction. The determinant reduction phase 
converts a matrix with polynomial elements and with a determinant which 
is also a polynomial in s, into another matrix which still has polynomial 
elements but which has a unity determinant. 

(3) Element order reduction. This phase operates on a matrix 
of polynomial elements having a unit determinant until a numerical ma- 
trix is reached. 

To illustrate the central ideas of this method, a 2X2 example of a simple 
yet non-trivial case of matrix factorization will be solved. Then, the 
general case will be examined and each step justified. 



EXAMPLE 



Suppose a simple two-dimensional system is given by 



GKs) 




1 

s + 2 
1 

s + 4 
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and is excited by two uncorrelated unit -valued white noise sources. The 
matrix of output power density spectra is 



£ w (s) = GK-s) . G T (s) = 



/3 



II 



(- S+2) (-S+3)(s+ 2 ) ( SH ) (- 5 + i) (- 5 + 3 ) (st i) (s + 4 ) 



| | 



-2^ + 17 



(-5+1 ) (-SH) (s+l) (5+3) (- 5 + 1 ) (-5+4 ) ($+ 1 ) (5+ 4) 

The inverse of the matrix G(s) is 

1 1 



G(s) 



-1 _ (s+l) (s+2) (s+3) (s+4) 

(- 4s + 10) 



s+4 

1 

s+l 



s+2 

1 

s+3 



which is unstable or unrealizable. Thus, the question is posed: Can a 
matrix G(s) be found which is realizable and inverse realizable, and is a 
solution to 

G(-s) G T (s) = § (s) = 35°<s) ? 



w 



(1) Pole removal phase 

The objective of this phase is to remove all the poles in every 
element. This is quite easily done by row and column multiplication. 

( -s+2) ( -s+3) 0 



T^-s) = 



( -s+l) (-s+4) 



(^(s) = T^-s) <g°(s) T^s) = 



- 2s + 13 



- 2s +11 



2s 2 +11 - 2s 2 + 17 



(2) Determinant reduction phase 

In this phase we first desire to manipulate the<£ matrix so it has 
a determinant which is constant and independent of s. This will be the 
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case if |T 2 (-s) | . | T g T (s) 



1 (s) 



Hence, 



If 



16 s + 100 = (-4s +10) (4s + 10) 



t 2 (-s) = 



1 

0 



0 

1 



-4s+10. 



^ 2 (s) = T 2 (-s) $\s) T 2 T (s) = 



2 s + 13 



2 s + 11 
- 4s + 10 



- 2 s + 11 
4s + 10 



- 2 s + 17 
(-4s+10) (4s+10) 



It is now desired to remove the poles in ^5 2 (s) without affecting 
its determinant. An adding transformation is thus called for. 



£> 21 (S) 



- 2 s +11 

- 4s + 10 



. 5s + 1. 25 



.375 
-s + 2. 5 



__ 2 k 

If the first row of (£ (s) is multiplied by 



s+2. 5 



and added to the 



second row, the pole will be cancelled if 

k . (-2 s 2 + 13) 

s = 2.5 
or k = . 75 

The total added quantity will be 
* 75 ( - 2 s 2 + 13) = 1.50 s + 3. 75 + 



+ .375 



375 



-s+2. 5 



s + 2. 5 



T 3 (-s) 



1 

.75 

-s+2. 5 
2, , „ T, 



§'<S) = Tg(-s) (jj (s) Tg (s) = 



- 2 s + 13 
2 s + 5 



2 s + 5 
2 
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T 2 (-s> 



1 

-4s+10 



| T 3 ( - S > 




T 

T 3 (S) 



(-4s+10) (4s+10) 



1 

4s+10 



) T 2 <S) 

1 



(3) Element order reduction phase 

Consider the array of the highest-order powers of s in each 

3 

element of <E (s) : 

-2 s 2 -2s 



2 s 



2 



Note that the first row is equal to the second row multiplied by -s. 
This is no accident, and arises because the determinant is independent 
of s. Performing a reduction transformation. 



T 4 (-s) 



1 

0 



s 



1 



$ 4 <s) = T 4 (-s) . <£ 3 (s) . T 4 T (s) = 



13 

5 



5 



2 



(4) Solution 

This numerical matrix is, oddly enough, the example considered 
in section 3. 3, for which the canonical triangular factorization is 



T 

N . N - 


o 

1 




F iw~ 


_ — — 


5 1 




n i 




J’H' 13 




0 "7T3 


From equations 3. 5 and 3. 6, 






"I v -1 


-1 v 


-1 




G(s) = T x (s) . T 2 (s) . 


T 3 (s) . T 4 




(s) . N 


G~}s) = N" 1 . T 4 (s) . T 3 (s) 


. T 2 (s) . T^s) 
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G(s) 



1 



5s + 13 
(s+3) (s+2) 




5s + 11 
(s+1) (s+4) 



s 

(s+3) (s+2) 

s + 10 
(s+1) (s+4) 



-1 

G(s) 



1 

52 (s+2. 5) 



(s+10) (s+2) (s+3) 

- (5s+ll) (s+2) (s+3) 



- s (s+1) (s+4) 
(5s+13) (s+1) (s+4) 



This example has illustrated the significant features of the general 
factorization procedure: 

(1) Poles removed by row and column multiplications. 

(2) Factors removed from a determinant of a polynomial matrix 
through successive introduction and removal of the inverse factor as a 
pole. 

(3) Reduction of a unit-determinant polynomial matrix by operat- 
ing on the highest powers of s in each element. 



The general nxn case will now be examined. 

( 1) Pole removal phase 

In the previous example, all RHP poles were identical in a single 
row, and all LHP poles were identical in a single column. This configura- 
tion facilitated the efficient removal of these poles by row or column mul- 

"th 

tiplications, but did not occur coincidentally. In the general case the ij 
element of ^ (s) is G.(-s) G.(s)| , where G,(s). is the k** 1 row of G(s). 



til 

All elements of the i' row of 0 (s) will have the same RHP poles, 

th 

which are the poles of^G^(-s)^, and the LHP poles in the j column of 
^(s) will also be similar, except for occasional cancellation effects 
in both cases. 
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(2) Determinant reduction phase 



The resulting matrix, which has elements which are poly- 

nomials is s, must have a factorable determinant with the RHP factors 
a mirror image of the LHP factors about the jw axis. If not, the random 
process is not realizable according to the discussion of section 3. 2. Con 
sidering the RHP factors, there is in general a constant and a number 
of not necessarily distinct zeroes in this determinant. 



Suppose that one determinant factor, -s + a, is selected. The 
transformation matrix 



T (-s) 



1 0 
1 



0 



1 

-s+a 



divides each element of the last row by -s + a. Let each of the resulting 

last row terms be expanded by partial fractions. The residue of the pole 

term in the nj** 1 element is ^^(a). The important question now under 

consideration is: Can each of the first n - 1 rows of $<•) be multiplied 
k. 



by a term i 

“S+cl 

ously all the poles in the last row? 



and added to the last row so as to eliminate simultane- 

k. . <jj ..(a) 



The added pole from the row in the column is 



ij 



-s+a 



Accordingly, the equation to be solved for n - 1 values of k^ is 



/*-/ 

£ 

^=1 



k. 

i 



^^(a) = - (j = 1, 2 n) 



This in effect requires that the last row be a linear function of 
the first n - 1 rows of C^(a). Since I f <a) [ = 0, because - s + a is a 
factor, the last row of ^(a) is always a function of at most the first 
n - 1 rows and the above equations can always be solved. 

The n - 1 element vector k is found from 
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£ n -l (a> k ] 



^ ,(a) 

nJ J 



where ^ ^(a) is the square matrix of the first n - 1 rows and columns 

of ^(a), and 5> .(a)~I contains the first n - 1 elements of the n^ row of 
__ - — n J J 

(a) . 

The pair of premultiplication transformation matrices is thus 



O 



i|_ K ... fc»-i 



O 



O 



o J_ 

-5+ A. 



I 

-S-ta. 



O 



I 

O 






o 

k*\- 1 



-S'tfa. Sifr. 



O 



• i > k^-i | 



From a computational point of view, kj should be determined 
and first used to transform the polynomial matrix with the right hand nu- 
merical matrix in the last expression above. Then, the - s + a factor 
should be removed from each element in the last row by synthetic division. 

The same transformation, only with the transposed LHP matrices 
in post-multiplication, will remove the s + a term from J([)"(s)|, Thus, the 



order of 



IS. 



(s) 



has been decreased by two. This procedure can obvious- 



ly be iterated for all factors, single or repeated, until the determinant 
is only a positive constant K. Then, multiplying the last row and column 
by ^ ^ will produce a matrix with polynomial elements in s and a unit 
determinant. 



The only case in which the procedure will not be applicable is when 
the last row and column of (j) (s) is zero except for the diagonal element. 
But in this configuration, the diagonal element can always be factored 
and the problem immediately degenerates to an n - 1 factorization prob- 
lem. 
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In summary, it has been demonstrated that a polynomial matrix 
can always be reduced by simple transformations to a form which has 
a unit determinant, independent of s. This is an original contribution 
to the general theory of matrices with algebraic elements, and is the 
key to the solution of the matrix factorization problem. 

(3) Element order reduction phase 

The starting point of this phase is a matrix having a unit determi- 
nant and the goal is to produce by successive transformations a numerical 

matrix. An algebraic matrix having a constant determinant is called in 

22 

the monumental work of Cullis an "impotent" matrix. 

Cullis proves that any unit -determinant impotent matrix can be 
obtained by successive multiplying-and-adding transformations on an 
identity matrix. Since the inverse of these transformations always exist, 
this means that there exists at least one set of transformation matrices 
which can operate on the given impotent matrix to achieve the identity 
matrix. Unfortunately, no method has been previously presented for 
determining this sequence but the procedure to be given next appears to 
be completely general and achieves the desired reduction. 

Suppose an array is formed of the highest powered terms in s of 
each element. Obviously, the terms in the determinantal expansion which 
have the highest power of s will all be formed from these terms and must 
sum to zero because the determinant is independent of s. In this array 
identify the terms which make up the highest power of s in the determi- 
nant. Replace the other terms in the array by zero. For example, suppose 
the highest terms are 

Is 4 -2 s 3 3 s 2 

3 2 

2 s -4s 6 s 

3 s 2 -6s -2 s 2 

g 

The highest power of s in the determinant is s . Replacing the 

g 

terms not involved in the s term by zero. 
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0 



1 s 



4 



-2s 



3 



2 s 



3 



-4s 



2 



0 



0 



0 



2 s 



2 



The determinant of this matrix must be zero, so one row can al- 
ways be expressed as a function of the other rows. In other words, a trans- 
formation can be readily found which will reduce the highest power of s 
in the determinental expansion. In the above example, this transformation 
is obviously performed by multiplying the second row by - -^-s and adding 
it to the first row. 

Iteration of this reduction of the highest ordered terms can be 
continued until no element contains a power of s. 

In the special case of the 2X2 impotent matrix, the determinant 
of the highest powered terms of all four elements is always equal to zero, 
and thus a series of simple operations of multiplying one row and adding 
it to another will speedily reduce the 2X2 matrix to numerical form. 

(4) Solution of the numerical matrix 

The only requirement that a solution exist to the factoring of the 
resulting numerical matrix is that it be positive definite. In the preceding 
steps, the factorizability of the determinant was the only realizability 
criterion needed. If one of the original diagonal elements had not been 
factorable, this would not in general have impeded any of the steps up to 
this point even though it would indicate an unrealizable system. However, 
referring to the matrix factorization procedure as a succession of linear 
systems operating on the random process, as was discussed in the be- 
ginning of this section, it is obvious that "unrealizability" and "realiza- 
bility" are both properties of a set of signals which are not affected by 
passage through a linear system. Therefore, a positive definite numerical 
matrix will result if the original power density spectra matrix satisfied 
the realizability criteria. A non-positive definite matrix implies a set 
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of white noise having imaginary auto or cross correlation. 



Appendix II gives a complete solution to a more complicated 3X3 
factorization example. 



This section has presented a general method for factoring a matrix 
of power density spectra, providing that the statistics arise from a multi- 
dimensional random process observable in the real world. Alternately, it 
has been proven that a linear multi -terminal system, excited by white 
noise, can always be found which (1) is stable, (2) has a stable inverse, 
and (3) reproduces the observed statistical interrelationships in a random 
process. 

3. 6 Matrix factorization: An iterative solution 

The method presented in the previous section is always valid, and 
invariably leads to an answer which satisfies all requirements. This sec- 
tion discusses an iterative procedure which will often yield a valid and 
speedy solution without the need to determine and factor the determinant 
of ® (s). This becomes especially valuable when the dimension of £w<s> 
is high, and when digital computers are used. 

The pole removal phase of the general procedure is readily accom- 
plished, and the real factorization problem deals with the resulting ma- 
trix with polynomial elements. Let this matrix be designated as (]5 (s), 
which can be expressed as a power series in s with numerical matrix 
coefficients 



The problem considered is to find a matrix H(s) which satisfies 
the equation 



Summary 









(3.7) 



H(-s) . H T (s) = ^ (s) 



(3.8) 
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where 



/vn 

H(s) = ^ 

k=o 



s k H, 



(3.9) 



Aw 



AW 



H(-s) . H (s) = 



2 <-l) k s k H, ] £ 



&=o 

Equating coefficients. 



— £J V J-o 



H. 

J 



£*• * £ <"D k 
k — 



H 



r- k 



where the range of k is bound by 



O fr ^ — /> >'V 






(3. 10) 

o — r~ k - a^w . 



The matrix factorization problem is, as an alternate interpreta- 
tion, 2 m + 1 non-linear matrix equations. Suppose an approximate solu- 
tion, H(s) , is known. If a small perturbation in H(s) is made with <jH(s) 
and the resulting product is to equal (s). 



- £ <-i>* 



< H k + dH k> (H r-k +dI V k> 



Neglecting the product dH^ • d H ^ ^ as of second order, the 

proper perturbation of H(s) is given by solution of the linear equations 

Sr ~ ^ ^ r-K ~ Hr-ft + Jj* d Hr-k (3.11) 

The left-hand side of this equation is recognized as the matrix 

coefficient of the IT"* 1 power of s in the error: <£>(s ) - H( -s) . H T (s) . 

th 

After these equations are solved for cjH^, the remaining f error will 



be 



£ dH 

k — 






r- k 



and the procedure may be iterated until the error becomes negligible, 
providing that the original guess was "close enough". 

Besides needing an approximate solution to commence this pro- 
cedure, another drawback is that the resulting solution H(s) is not guar- 
anteed to have a realizable inverse -- that is, H(s) may contain RHP 
factors. To handle both of these requirements, a good initial solution 
for H(s) will often be the LHP factors of the diagonal elements of ^(s). 
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This first trial, while obviously in error if there are any non-zero off- 
diagonal elements in ^ (s), will be close to the solution if the cross- 
correlation among the signals is weak. Also, it definitely has a deter- 
minant which has all LHP factors, which a small perturbation in the co- 
efficients of H(s) will not appreciably modify. 

Having some promise of solving the matrix factorization prob- 
lem with successive linear equations, it is useful to consider these equa- 
tions in more detail. The set of equations to be solved is, from Eq. 3.11, 



e 

r 



(- 1 ) 



k 






H r-k + H k 



O ± k £ /**- 

M. 



~~ £ 



where 0. is derived from 
- c r 



rlH T \ °- Ri/ 

2 H r-k J 
(r= o, i , , , 2/>w) 






§> (s) - H(=s) H T (S) = ^ 5 

— — ilr= o 









The new H(s) equals the original H(s) plus dH(s). 



The total number of independent variables is the number of in- 
dependent elements of 3L. For r even, where {£0 is symmetric, these 
are the diagonal and above -diagonal elements. For r odd, where is 

skew-symmetric with zero -valued diagonal elements, these are the above - 
diagonal elements. The total number of independent elements is thus 



(m + 1) (n) (n + 1) + (rip)(n) (n - 1) _ + 1 ^2 n(n - 1) 



The number of unknown variables is the number of coefficients 

2 n(n *“ 1) 

of dH(s), which is (m + l)n . Therefore, ^ elements of d H(s) 

can be arbitrarily selected, which reflects the degrees of freedom of 
the imbedded numerical matrix in the complete rigorous solution. One 
way of removing this excess is to specify that 4 H q be symmetric. 

To illustrate these ideas and to indicate the expected degree of 
convergence, the sample problem solved in section 3. 5 will be re-solved 
iteratively. 

After the pole removal phase. 
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^ (s) = 



elements of ^ (s) 



H(s) = 



- 2 s 2 + 13 


- 2 s 2 + 11 




- 2 s 2 + 11 


- 2 s 2 + 17 




solution for H(s) is the LHP factors of the 








1.414 s + 3. 61 


0 




0 


1. 414 s + 4. 


12 



H 





3.61 


0 n 




1.414 


0 


= 






H 1 = 








0 


4. 12 


1 


0 


1.414 



The equations to be solved are, from Eq. 3. 11, 



^ e 
2 



dH HT+ H dH T 
o ° o o_ 

T T 

dH H + H dH, 

o _J_ o 1 

T T 

* dH H - H . dHj. 



r T T 

dH, H - H, dH 
1 o 1 o 



^ e (s) = 



- 2 s + 11 



2 s +11 
0 



As an example of the appearance of these equations. 



$ 6 = 


S GD 




*11 *!°2 




3.61 O' 




3. 61 0 




r O Ol 

dh ll dh 12 


W o 




- 








+ 










_u od| 




- dh 12 *22. 




0 4. 12 




0 4. 12 




- h 12 dh 22__ 



where dH Q was selected as symmetric. The boxed elements of ^ in- 
dicate a set of independent equations. This set of equations can be solved 
directly, yielding dh^ = 0, dh^ = 0, dh® 2 = 1.424 , 

Solving the four remaining independent equations in and 

yields 



- 58 - 



. 660 s + 1.424 



dH(s) = 



. 753s + 1.424 



The new H(s) is thus 



1.414 s + 3. 61 



. 753 s + 1.424 



. 660 s + 1. 424 



1.414 s + 4. 12 



H( -s) H (s) = 



2.435 s + 15.03 



2 s + 11.02 



2 s + 11.02 



-,2. 568 s + 18.93 






. 435 s - 2. 03 

** .02 



- .02 



+ . 568 s - 1.93 



Repeating the solution of the seven linear equations, the new H(s) 
is given by 

' 1. 22 s + 3.285 ,7456 s + 1.5348 



.913 s + 1.5348 



2 



H(-s) H (s) = - 2.047 s + 13.15 



1. 128 s + 3. 844 



- 1.957 s - .012 s + 10.95 



1.957 s 2 + .012 s + 10.95 - 2. 105 s 2 + 17.16 



which is compared with the actual <]5(s) 
J) <s) = 



2 s + 13 



2 s + 11 



- 2 s + 11 



-2s +17 



This solution is probably within the accuracy of measurement 
of $<s>, and no further iteration is made. | h(s) | = . 697 s 2 + 5.862 s + 
10.66 which is stable. 
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In high order problems evaluating and factoring the determinant 
of H(s) can be a very difficult step. If an indication of inverse realiza- 
bility is desired, an approach similar to that used by the Nyquist stability 
criterion is very useful. Jh(jco) | * s eva ^ uate< ^> possibly by a digital 
computer, for a sequence of various values of w and plotted on a complex 
plane. The presence of RHP factors will then be detected by any net num- 
ber of encirclements of the origin. 

To summarize, the described iterative method presents an attractive 
alternative to the complete factorization procedure, especially when digital 
computation is employed. In an example of this method, two iterations 
solved the problem to an acceptable accuracy level. The price which must 
be paid for this computational advantage is the possibility of a non-con- 
verging solution or one which converges on a solution having an unrealiz- 
able inverse. 



3. 7 Matrix factorization: A lightning solution 



This section considers a very special case of matrix factorization, 

but one which is quite simple to solve. The central requirement is that 

T 

each non-zero element of any single row of G(s) , where G( -s ) . G (s) = 
5^(8). must have separate and distinct poles and must have a deno- 
minator of higher order than the numerator. 




The first question to be considered is whether each of the n terms 

. Alter- 
can all 

n ( a\ T 

poles belonging to a single 



p / s \ 

G ;1 (-s) G., (s) can be recovered from a knowledge of ij 

+ i -e + . , , + . f P..(s) . . Q..(s) 

nately, if a partial fraction of ij is made, ij 

^i/ S ^ element of G(-s) or G^(s) be 



grouped together, and can these groups be further separated into LHP-RHP 
product pairs ? 



The key to this grouping is that any scalar function A(-s) B(s), 
where A(-s) and B(s) have RHP and LHP poles respectively, has an in- 
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verse time-domain transform TU / which is continuous across the origin 
as long as the order of the denominator is at least two degrees higher 
than that of the numerator of A(-s) B(s). This can be proven by showing 
that f(0 + ) = f(0 ) which, by using the contour integral to sum residues, 
becomes 

JO® JoO 

2 *j J ds A(-s) B(s) = J~ds A(s) B(-s) 

-J- ° 

The latter equation is valid since the right hand side is merely the 
left-hand side with the sign of the integrating variable changed, as the 
negative sign of the differential is cancelled by the limit exchange. 



As an example, let 

A(-s) B(s) = 



(- s + 7) (s + 2) (s + 4) 



sm .± e’ r (^°) 



1 - 2T 3 - 4T 

-7T- e 



22 



(n») 



«o -) . f(o + ) . -i- 



Thus, the residue of the LHP poles must sum to those of the RHP 

poles in the partial fraction expansion of any such function as A(-s) B(s). 

p (s) 

Therefore, the partial fraction expansion of — ^ - can be grouped 
to show this residue equality between, in general, n se*ts of LHP and RHP 
poles, providing that all elements have distinct poles. If i = j, each LHP 
pole has an equal RHP pole in the partial fraction expansion, and this 
grouping is impossible. 

Suppose that n such sets of RHP poles have been determined in 

one element of the first row of ^ (s). Under the assumptions of the 

vv 

form of G(s), these sets should satisfy residue equality requirements 
in every off-diagonal element of the first row of ^^(s). The first diagonal 
element is similarly grouped, and the corresponding LHP and RHP terms 
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of each set are multiplied together. The resulting n terms are ^ .(s) = 

^ GA- 5 )G. .(s), and thus when individually factored, yield the first row 
j= i J y 

of G(s) which may be placed in any desired order. 



th. 

Having fixed the first row of G(-s) , given by^G^(-s), ^the j element 

of the first row of (s) is>G..(-s) . G.(s) , and thus G.(s) can be found 

w I * J — I J 

directly for all j, since the residue equality requirement associates each 
element of G.(s) with the known set of poles in an element of,G 1 (-s). 



L_L 






G(-s ) G (s ) is then evaluated, and under the restrictions of distinct poles 
of G(s), will equal ^ (s). 



w 



As an example, the much-battered veteran of this chapter will 
be resolved. 



2 s + 13 



2 s + 11 



w 



(s) = 



(~s+2) (-s+3) (s+2) (s+3) (-s+2) (-s+3) (s+1) (s+4) 



- 2 s + 11 



2 s + 17 



( -s+1) (-s+4) (s+2) (s+3) ( -s+1) (-s+4) (s+1) (s+4) 



/ \ _ -2s + 11 

1 2 (-s+2) ( -s+3) (s+1) (s+4) 



-s+2 -s+3 



s+1 



s+4 



(-s+3) (s+1) 



(-s+2) (s+4) 



The poles at s = 3 and s = 2 satisfy separate residue equality 
requirements, lending support to the hope that ^ ^ (s) can result from a 
G(s) with distinct elements. 



w 



V V (s) = 
Let G,A-s) = 



- 2 s + 13 



1 1 (-s+2) (=s+3) (s+2) (s+3) 

1 



11 ' 



-s+3 and °12 ( - S) = _ -s+2 



(-s+3) (s+3) (-s+2) (s+2) 

1 



I VjV 2 (s) = G n (-S) G 21 (s) + G 12 (-s) G 22 (s) 
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1 




1 



s+1 



s+4 



and the resulting G(s) is given by 



D 



1 



G<s) = s+3 



s+2 



1 



1 



s+1 



s+4 



and G<-s) G^(s) yields the given ^^(s). 



But the problem is not yet complete, and this example was pur- 
posely chosen to illustrate a significant defect of this simplified attack. 

As given in section 3.5, G(s) contains a RHP pole and is unrealizable. 
Generally, the resulting solution in this method may or may not be in- 
verse realizable, but its simplicity makes the attempt worthwhile as a 
preliminary to the increasing rigor, generality, and computational 
complexity of the methods given in sections 3. 6 and 3.5. 

3.8 Statistical degrees of freedom of a multi -dimensional random process 
Up to this point it has been assumed that 0^(s) is a non-singular 



hypothesized nxn G(s) is a linear function (not necessarily numerical) 



sized transform of the white noise excitation vector over a finite interval. 



Therefore, v^s) is a redundant member of the set of signals and 
can contribute no additional statistical information on the multi -variable 
random process. At this point the representation of G(s) as an nxn matrix 
excited by n uncorrelated white noise sources is open to question, since 
there are less than n "useful" outputs. 



nxn matrix. If 




0, this implies that one or more rows of a 




Vk<*)= 1 C X (S) ,£<<?), w»>] = £ £*(?) icuo 
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Suppose, by striking out pairs of rows and columns, the highest 
order non-singular matrix contained in ^^(s) is found. Denote this ma- 
trix as representing a set of m independent components of the 

set v. It has been shown in this chapter that if physical realizability 

criteria are satisfied ^ (s) can be factored into G (-s) . G ^(s), with G (s) 

^m m m m 

excited by m white noise sources. It appears logical that the remaining 
n - m dependent signals can be derived from these m white noise sources, 
as shown in Figure 3.1. 




Figure 3. 1. Formation of a multi -dimensional random process 
with redundant elements. 



The adequacy of this model will be proved in the following steps: 



a m ^ s ) will be found which satisfies the cross power density spectra 



relationship between every v. and v.. It will then be shown that this H (s) 

k l n-m 



produces signals v which have the proper cross power density spectra 

.K 

among themselves. Thus every signal will be related as indicated in the 
original ^^(s) matrix and Fig. 3. 1 will indeed be a valid representa- 
tion of a multi -dimensional random process with a matrix $U< S > Of 



rank m. 



To better picture the following steps, ^ vv (s) is shown in parti- 



tioned form. 
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5 i*) _ 



' ^1T. iTio ) 






StT^VrC?) 

From Eq. 2. 32, 

? v i v k < s ) = G_(-s) hJ 1_ _(s) 



? 

o 



Hvi -sm (r ^ ) 



&Z>>; 

l 

(3.11) 



m 



n-m 



H 



n X (S) = G *(-s) ^V.V.(B) 

n-m m ^ x k 



(3.14) 



-l 



[ G m ( - S) l exists because of its non-singularity. But also, ^(s) niust 
satisfy the equality. 



v, v (s) = H (-s) . H (s) 
k k n-m n-m 



From Eq. 3. 14, the following relations must hold for the parti- 
tioned sub-matrices of ^ (s) 

. vv 

£ 



V V 

kk 



(s) * § v.v k (-s) [G m ' 1 <s)] T . [G m _1 <- S )]. f v.v k (s) 



= ^V k V.(s) f^s) 5v.V k (s) 



(3.15) 



Since (s) is of rank m, each of the last n-m rows can be 

X W /Kv 

considered as a linear function of the first m rows. Let (s) = A. .(s) 
_ 1 * I -i=| To. 

W .(s) where ,<]>, (s) and <l>.(s) are row vectors of $ (s) and A. .(s) 

I i i rk i i~*~ i i w Tu 

is a scalar to be determined. Writing this equation in complete matrix 
form, and recognizing the resulting partitioned matrices. 





[^ v k v i (s) ! 


^ V lr V lr 

k k 


(s) ] = [ 


A < S >]j$ m <8) | Vk (S) ] 




^Vi (s) = 


A(s) 


^ m < s > 




and 


® v k v k (s) = 


A(s) 


S v i v k (s) 






A(s) = ^ 


Vi is) 


* m 
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^ V V (s) = ^ v v.(s) ^ 1 (s) ^ v.v (s) 

k k k 1 m l k 

Thus equation 3. 15 is verified, providing that some matrix A(s) 

exists, and the assumed form for H (s) produces the observed sta~ 

n-m 

tistics. Note that H (s) is fixed for a choice of G (s) . Transposing 

n-m m 

Eq. 3.14, 

H (s) = $ v v.(-s) [ G " r (~s)~] 

n-m k l < - m - 1 

= A(-s) § <-s) [g^-sTT 1 

m L m A 

= A( -s) G <s) G T ( -s) fiGM-sfl" 1 

m m L m J 

= A(-s) G (s) (3.16) 

m 

For m ^ s ) to physically realizable, A(s) must contain only RHP 
poles. A(s) was used as a row transformation to express the redundant 
rows as a function of the independent rows. The elements of A(s) can be 
used in an elementary transformation at the beginning of the factoriza- 
tion problem to eliminate all redundant rows and columns, leaving 
Physically, this means that the random process v is passed through a 
matrix filter B(s) , such that the resulting output power density spectra 
matrix 

= B(-s) $ (s) B T (s) = 



I 

I 0 

L 

! ij 

That is, B(s) weights and adds together the m independent signals 
of v and nulls out the redundant signals. As discussed in the first para- 
graph of this section, this dependence among signals, if observed in a 
stable random process, must arise in a physically realizable system. 
Therefore, B(s), containing all the elements of A(-s), must be physically 






where B( -s) = 



-A(s) 
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realizable or ^^(s) does not represent a real process. 

Thus an index of randomness of a multi -dimensional random 
process is the rank of the matrix of cross -spectra or, alternately, the 
number of white noise sources needed to reproduce the statistics of the 
process. Also, a set of dependent random processes is physically rea- 
lizable only if the redundant rows of the matrix of power density spectra 
can be removed by a row transformation with RHP pole factors. 
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CHAPTER IV. 



NEW RESULTS IN OPTIMUM SYSTEM THEORY 
4. 1 Introduction 

The previous chapters have been in a sense an introduction, although 
a useful one, to the main theme of this report. It has been demonstrated 
that a linear system excited by white noise can always be found to dupli- 
cate the basic statistical properties of any stationary random process, 
single or multi -dimensional. 

In standard texts on random processes it is customary to note 

that a power density spectrum has the same form as the spectrum of the 

output of a linear filter excited by white noise. In the early paper by Bode 
4 

and Shannon , which served to convert the highly mathematical approach 
of Wiener into a form more understandable to engineers, this white noise 
filter and its inverse were used as a means to remove all memory from 
the random process and to justify the use of a straight<3^ * operation to 
obtain the optimum configuration. In this work the idea is carried one 
step further and the hypothesis is offered that within the confines of a 
linear theory a random process should be viewed as actually being the 
result of white noise exciting a linear system. Although this system in 
some cases cannot be physically represented and the white noise sources 
cannot be traced to microscopic random phenomena, it is possible to 
make measurements on the random process itself with complete mathe- 
matical assurance that there is such a linear system "upstream and 
around the bend". 

This hypothesis would be only of mild interest by itself, but this 
chapter will show how this simple assumption makes the study of sta- 
tionary random processes purely a measurement problem and how it tends 
to unify the conventional analysis techniques of linear systems and those 
of stochastic processes. 
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4. 2 Matrix differential equations and system state 



The heart of the description of a linear physical system is its 
"state", which effectively describes the condition of every internal en- 
ergy storage element at every instant. Since a random process is to be 
analyzed in terms of its equivalent system, it is useful at this point to 

summarize the major features of the matrix theory of differential equa- 

20 

tions, such as is found in Bellman , in order to emphasize the state 
approach to the analysis of linear systems. In this case, matrices allow 
compact expression of ideas without regard to order and dimensionality 
of the system under consideration. The standard theory outlined in this 
section will provide a foundation for clear presentation of the original 
results to be presented in the remainder of this report. 

The basic matrix representation for a linear system is presented 
in the following equation 

-±- x = A x + D u (4. 1) 

dt 

where x is the n-dimensional state vector of a linear system, A is a 
constant nxn matrix, D is a constant nxm matrix, and u is the m-dimen- 
sional excitation vector. For example, consider the simple second-order 
system of Figure 4. 1, where a spring-mass-dashpot system is being ex- 
cited by an external force F. 




Fig. 4. 1 A simple second-order system 

Here the differential equation is 

M Hi + B H + K x = F 

dje 35- 

dx 

Defining one state variable, x^, to be x, and x to be , is 
sufficient to fix the potential and kinetic energy of the system. The sys- 
tem equations are next cast into the general form of Eq. 4. 1. 
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The initial condition response or free behavior of linear systems 
will be of particular importance in the study of random processes in 
following sections. Given x (0), it is desired to find x (t) under condi- 
tions of no external excitation. It would obviously be desirable to have 
a solution in the form 

x (t) = B (t) x (0) 

Assuming this form and substituting into Eq. 4.1, 

B (t) x (0) = A B (t) x (0) 



_d_ 

dt 



or 



The series 



B (t) = A B (t) 



where 



B (t) = I + At + A 2 y 
B (t) = A + A 2 t . . . . + A n t 



+ A 



n t 



n 



n-1 



n! 

= A B(t) 



dt ~ (n-1)! 

satisfies this equality. 

Thus, 

^2 n 

B (t) = A n — ■. , where A°= Lis the desired 

/n= ° n! At 

solution and is known as the matrix exponential e , a quantity that is 
convergent for any value of A and t. It is analogous to the scalar ex- 
ponential, and occupies a position of pivotal importance in linear systems 
analysis. 

If Eq. 4. 1 is Laplace transformed, 

s x(s) - x (0) = A x (s) + D u (s) 

-1 r . .-7-1 



(s) = j^s I - A^J x (0) + [[si — A~j D u(s) 

(4.2) 
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The eigenvalues of the matrix A are thus the pole locations of 
the transform of the transient response. If, for example, A has only 
diagonal elements \ , 

[--r ■ [ttK- Jy 

x^(t) = x^(0) e ^ (4.3) 

In this case, the state variables refer to a system which, in La- 
place transform terms, has been expanded by partial fractions into a 
series of simple poles. There is no unique state of a system, since any 
non-singular linear transformation can be made on a particular set of x. 
If x = T y, substituting in Eq. 4. 1 yields 

T hf y = AT y 

I- y - T' 1 AT, 

A transformation on A, where T ^ AT becomes a diagonal ma- 

20 

trix, is always possible if A has distinct eigenvalues . In this case, 
the general solution for a free system is, from Eq. 4. 3 

y(t) = [e ^ <f l y (0) 

1J J 

T _1 x(t) = [e^ cf. 1 T" 1 x(0) 

1;J _j 

x(t) = T [e* SJ T _1 x(0) 
u ij J 

Therefore, 




for any A which has distinct eigenvalues V and which is reduced to diag- 
onal form by T. In the general case, from Eq. 4. 2 

e At = $ -1 £ s I - Aj _1 (4.4) 

An alternate way to visualize the concept of state is to integrate 
and Laplace transform the basic equation, Eq. 4. 1, yielding 



x. (0) 

and x.(s) = r — 

i s -A . 
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